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Abstract 

This paper reports on an on-going research and development effort in remote surface 
inspection of space platforms such as the Space Station Freedom. It describes the space 
environment and identifies the types of damage for which to search. This paper provides an 
overview of the Remote Surface Inspection System that has been developed to conduct 
proof-of-concept demonstrations and to perform experiments in a laboratory environment. 
Specifically, the paper describes three technology areas: 1) manipulator control for sensor 
placement, 2) automated non-contact inspection to detect and classify flaws, and 3) operator 
interface to command the system interactively and receive raw or processed sensor data. 
Initial findings for the automated and human visual inspection tests are reported. 

Introduction 

Space platforms such as the Space Station Freedom (SSF) are complex and expensive 
scientific facilities which must operate in harsh and remote environments. Such facilities 
must be maintained, i.e., inspected, replenished and repaired periodically to assure safe 
operations for the crew and to provide reliable experiments and apparatus for scientists. 

Although inspections and repairs may be performed by astronauts, there is considerable risk 
and cost in Extra Vehicular Activities (EVA). Studies have shown that astronaut EVA time is 
over-subscribed for SSF maintenance and therefore whenever possible tasks should be 
performed by means other than EVA. In addition, the Freedom will be unattended 87% of the 
time for the first few years. This precludes any EVA activity except during Shuttle visits, when 
experiment tending and scientific research have greatest emphasis. Given the state of 
robotics technology, non-contact inspection leading lo preventive maintenance, as opposed 
to breakdown repair, presents a cost-effective opportunity for the use of 
robotics/teleoperation in space. The recent “Space Station External Maintenance Task 
Team” report identified inspection as the most tedious and time-consuming task that needs 
automation [1]. 

Due to the SSF's large size, it is not logistically practical to mount fixed observation cameras 
and other sensors to cover the entire facility. Telerobotics is therefore a natural technology 
for remote visual inspection of the Space Station and other large space platforms. 

This paper describes the research and preliminary results on remote surface inspection and 
provides a brief review of the newly developed Remote Surface Inspection system at the Jet 
Propulsion Laboratory. The organization of this paper is as follows: Section 1 provides a 
brief background on the space environment, including highlights from the Long Duration 
Exposure Facility (LDEF). Section 2 furnishes information on the Space Station Freedom 
and identifies inspection tasks required for its maintenance. The Remote Surface Inspection 
system is described in Section 3 where its architecture and capabilities are outlined. Section 
4 describes our current approach to both human visual inspection and automated 
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inspection. In Section 5, some issues in machine-vision-based inspection are discussed. 
Experimental results and conclusions are presented in Sections 6 and 7, respectively. 


1. SPACE ENVIRONMENTAL EFFECTS 

The low earth orbit environment consists of many hazardous elements that may lead to a 
degradation of material properties and threaten the surface elements of satellites and space 
platforms. Many factors such as exposure to mono-atomic oxygen, solar wind, ultraviolet 
rays, radiation reflected from the earth, thermal cycling effects due to the motion in and out of 
the earth's shadow, meteoroids, and space debris will cause damage to varying degrees. 
Past space missions and in particular the Long Duration Exposure Facility (LDEF) have 
shown that severe damage and degradation can result to exposed and protected flight 
hardware surfaces, particularly to organic materials, thin foils, and certain other coatings, 
when the exposure is for an extended period of time [2]. 

The LDEF spacecraft was a 30 foot long, 14 foot diameter, 14-faced (12 sides and two ends), 
open-grid structure on which a series of 86 rectangular trays were used to mount experiment 
hardware. The spacecraft was exposed to the on-orbit space environment for about 6 years, 
through a 257 kilometer altitude orbit, approximately the same orbit band that Space Station 
Freedom will utilize. The Meteoroid and Debris Special Investigation Group (M&D SIG) 
found more than 34,000 impact features on all space-exposed surfaces, most below 1.0 mm 
in diameter. Figure 1 shows several samples of LDEF material. Table 1 summarizes the 
types of features by size. The largest impact crater was 5.7 mm in diameter [3, p. 5] and 
probably caused by an object about one millimeter in diameter. The smallest crater identified 
to date [3] is 0.1 micrometers in diameter. 

Additionally a wealth of information pertaining to the constantly changing space 
environment and its impact on long duration spacecraft surfaces has been analyzed and 
reported [4]. Information directly relevant to design of telerobotic spacecraft surface 
inspection systems include: (1) how surface orientation (Earth facing, Space Facing, 
Leading Edge, Trailing Edge) alters the rate of surface impact or damage; (2) the flux rates 
(number-of-impacts/area/second) associated with man-made (debris) and natural 
(micrometeoroid) impactors; (3) the rates at which space debris and meteoroids are 
increasing in Low Earth Orbit; (4) the moiphological features of surface damage/impacts as a 
function of the physical form and composition of both the impactor and the impacted surface. 

On the LDEF satellite, the leading edge was mostly impacted by debris (or man-made 
impactors), while the trailing edge was mostly impacted by meteoroids (natural) [3]. A short- 
term increase in micro-particle debris impact (or flux-rate increase) following LDEF 
deployment has been observed and attributed to increased Shuttle activity during this 
period. On average, the impacts in the first year were higher that in the following years. 

The LDEF post-flight investigation is not yet complete. For current information and results, 
the reader is referred to the Environmental Effects newsletter, published by the LDEF 
Systems Special Investigation Group [5]. 
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Figure 1 (a) This pre-delntegration 
view shows one module of LDEF 
tray Hi 2 with the thermal blanket 
partially peeled back exposing the 
detector stack below. 


Figure 1 (b) A penetration 
hole through one of the 
HI 2 multi-layer blankets. 


Figure 1 (c) A concentrically-ringed 
impact feature into white-painted 
aluminum surface. 


The present average rate of increase of space impactors (debris and micro meteoroids) is 
around 5%/year. However, the LDEF information obtained so far has validated existing on- 
orbit micro-meteoroid and orbital debris (MMOD) models, which predict an approximately 
11% yearly increase in particulate contamination of the Earth-orbit region [6]. It is estimated 
that this number will increase to about 20%/year by the turn of this century [7]. These results 
have prompted design rules for Space Station that include avoiding use of organics and 
thin-film foils for the SSF external surfaces, in favor of anodized aluminum and Z93 coatings 
(flat black or flat white, as thermally appropriate). External structure is being designed to 
withstand the average on-orbit effects for thirty years. These are reasonable 
countermeasures to naturally-induced, detectable wear. 
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Such information as the models provide can not only be utilized to design future spacecraft, 
but can also be used to derive requirements for an inspection system. For example, the 
micrometeoroid impact features shown in Table 1 strongly indicate that in order to use an 
inspection system to revalidate future MMOD models, the system must be capable of 
detecting very small flaws in the range of 0.2 to 6.0 mm on surfaces of varying shape and 
specularity under orbital lighting conditions. This must take place while satisfying safe 
clearance requirements, as well as other requirements for mobility and safety such as 
smooth motion, collision free scanning, and so on. 


Table 1. Summary of LDEF Impact Features [2]. 


Feature Size 
(diameter) 

Clamps.Bolts, 
& Shims 

Tray 

Flanges 

Experimental 

Surfaces 

Totals* 

< 0.3 mm 


- 

2911 

3069 

> 0.3 mm 

- 

- 

763 

763 

< 0.5 mm 

1318 

1923 

19342 

27385 

> 0.5 mm 

161 

419 

2539 

3119 

Totals 

1479 

2342 

25555 

34336 


* Note: the "Total" is greater than the sum of the individual column entries for the "<0.3 mm" 
and the "<0.5 mm" rows because some of the features contributing to the total were detected 
on intermediate surfaces such as between the tray flanges and the experimental surfaces. 


2. SPACE STATION FREEDOM 

The Space Station Freedom (SSF) is a large space platform with complex mechanical, 
electrical, thermal, fluid and gas interfaces, and a changing suite of internal and external 
scientific experiment apparatus. Over a 30-year design lifetime, Freedom will be adapted 
and upgraded as our knowledge of the effects of prolonged microgravity exposure on living 
creatures and inanimate objects advances, and we identify new experiments and processes 
to perform. On-orbit maintenance of such a complex, changing facility requires periodic as 
well as "on demand" inspection capabilities. Although subjective "eyes-on" observations 
during planned crew-EVA will gather much important data, telerobotic inspection offers 
precise repetition of calibrated sensor placement and positioning, enhanced (non-visible 
light) sensing, digital scene recording and matching, and greater automation in flaw 
detection and categorization. 

Periodic inspection is required to ensure that potential problems are detected early on and 
changes in SSF external configuration and appearance are monitored. This type of 
inspection can be scheduled to take place non-invasively, e.g., when no other major activity 
is planned. On-demand inspections, for example in preparation for crew-EVA, can aid 
operations planning by assessing the condition of external SSF structure or interfaces at an 
EVA worksite. Revisiting previous worksites where work was suspended can determine if 
equipment left there is in order. Orbital Replacement Unit (ORU) installation site inspections 
can determine empty interface conditions, affecting tool manifesting for the next EVA visit. 
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Although at the present time the ground-based control of robotic devices is not part of the 
SSF baseline design, NASA is interested in performing a feasibility study to determine if 
future ground-based telerobotic operations can supplement on-board operations. In the 
several years prior to permanent manned operations, when the station will be mostly 
untended, ground-remote telerobotic operations could support both periodic and on-demand 
(in response to anomalies detected by dedicated sensors) non-contact inspections. These 
activities could provide useful, detailed preliminary information vital to planners who must 
make the most efficient use of limited crew-EVA during man-tended operations. 

2.1 Inspection Requirements 

Although at the present time, detailed inspection requirements have not been specified, 
NASA has emphasized the need for inspection in various documents and forums. For 
example, the Space Station Freedom External Maintenance Task Team Final Report (See 

[1] , Appendix E) specifies high-level requirements for the inspection of the station and states 
that telerobotics should be utilized to accomplish some of the inspection tasks, in particular, 
routine and repetitive ones. A number of candidate tasks have been identified based on our 
interactions with engineers at the Johnson Space Center (JSC) and various scientists 
working on LDEF. These include inspection of (1) truss strut damaged by micrometeoroids, 

(2) cracks in structures, (3) shield area damaged by micrometeoroids, (4) thermal blankets, 
radiators, or solar panels damaged by micrometeoroids and atomic oxygen, (5) 
thermal/mechanical interfaces at ORU installation sites, (6) deployable mechanisms for 
incorrectly positioned latches, connectors, and other mechanical devices, (7) the SSF shuttle 
docking port before each docking, (8) damaged fluid and power lines in a utility tray , (9) 
effects of fluid leaks on optics, and (10) magnetic fields, plasma fields, and contaminant 
levels, especially hydrazine concentration. 

3. TELEROBOTICS INSPECTION SYSTEM 

In this section we describe a telerobotics inspection system that has been developed to 
perform human visual inspection experiments in realistic space-like environments and to 
develop and integrate new supervisory robotic and automated inspection techniques. A 
strong emphasis has been placed on duplicating critical space environment effects within the 
available laboratory space and budget. This system consists of local and remote sites. In our 
terminology, a local site is where the operator resides, which can be the habitation module 
or a ground station on the earth; the remote site is the task location where the robot is and 
the inspection takes place. 

The remote site elements consist of one manipulator arm mounted on a mobile platform, 
which carries lights, cameras and other sensors for inspection and manipulation, and 
computers for processing real-time inspection data. Figure 2 shows the remote site of the 
system. The inspection task mockup board is a one-third scale of a section of the Space 
Station truss structure. Two tank ORUs are mounted on this truss structure to provide typical 
surfaces for inspection experiments. The manipulator is a seven-degree-of -freedom (7-DoF) 
Robotics Research Corporation™ (RRC) arm which is mounted on a one-degree-of-freedom 
motion platform. Since six DoF are sufficient for arbitrary placement and orientation of the 
end-effector, the "extra" DoF in the arm is used to provide direct control of the elbow position 
independent of the end-effector position and orientation. This is accomplished by using the 
"configuration control" methodology [8] developed at JPL to control the "arm angle," which is 
the angle between the arm plane and a reference plane. The control scheme provides 
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robustness to kinematic singularities and allows the user to specify weighting factors for the 
task requirements. The configuration control of the 7-DoF RRC arm with elbow positioning 
capability has been implemented for real-time control of the arm. 



Figure 2. Remote Surface Inspection System, showing 1/3 Scale Space Station 
Truss Mockup, ORUs and Inspection Manipulator Arm. 


The one degree-of-freedom platform provides base mobility to the RR arm and increases the 
workspace of the arm considerably. The mobile platform is treated as the eighth joint of the 
manipulator system. Following the configuration control approach, an additional (8th) task is 
defined to resolve this redundancy. The additional task is formulated as the control of the 
"elbow angle," which is the angle between the upper-arm and the forearm. When this angle 
indicates that the arm is over-stretched or under-stretched, the platform is moved 
automatically to restore the optimal condition, without perturbing the end-effector position 
and orientation. Thus the automatic motion of the platform prevents undesirable over- 
stretched or under-stretched arm configurations. 

Reference [9] provides details of the architecture, algorithms, and hardware description of the 
manipulator control system. The arm functionally simulates the Special Purpose Dexterous 
Manipulator (SPDM) and the motion platform provides translational capability which 
simulates the mobility of the SPDM provided by the Space Station Remote Manipulator 
System (SSRMS), in a limited sense. Realistic SSF environmental effects are provided to 
improve the operator's perception in executing arm functions. 
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Lighting and viewing at the remote site is achieved by means of a controlled environment, 
i.e., black ceiling, floor, and curtains. Two sets of lights are used in the laboratory. One 
simulates Low Earth Orbit sunlight and the other simulates the controlled environment 
lighting of the SSF. Figure 3 shows the 1200W, 5600°°K +/- 400°<>K, adjustable-focus Luxarc 
1200 lamp for Sun-like illumination producing high contrast between shadowed and lit 
surfaces. This lamp is mounted on a moving platform with computer-controlled pan/tilt 
mechanisms and intensity to mimic realistic analog changes in sun angle and strength as the 
SSF orbits the earth. Two controlled lights are mounted on the end-point of the manipulator 
arm to provide close-up illumination and to light enclosed regions. In addition, three other 
cameras with pan, tilt, zoom, and focus control capability are mounted in the laboratory to 
provide functions similar to those that will be available on the SSF. 



Figure 3. Computer-controlled, 
simulated solar illumination of the 
worksite is provided by this 
1200W lamp with variable pan, 
tilt, focus and X-travel capability. 


The local site provides operator interface hardware and software as well as a data 
logging/viewing and simulation facility. The control station is composed of three high 
resolution color monitors, a Silicon Graphics IRIS workstation, two shuttle-like joysticks, and 
a control panel that provides the camera, light, and video switch interface to the operator. 
Figure 4 shows the overall operator control station, housed in a Space Station cupola 
mockup that realistically simulates the equipment and operator space limitations. Figure 5 
shows a block diagram of the Remote Surface Inspection System 
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Figure 4. Remote Surface Inspection Laboratory, showing the Operator 
Control Station housed in a full-size cupola mockup of the Space Station. 


4. INSPECTION STRATEGIES 

Three complementary inspection strategies have been implemented. They are: 1) 
teleoperated human visual inspection, 2) automated scanning with human visual inspection, 
and 3) automated scanning with machine-vision inspection. The simplest and most reliable 
remote inspection technique is to present images of the area of interest directly to an 
operator and record those regions which the operator determines contain flaws. This 
technique relies on the operator to control the arm, lights, and cameras, in addition to 
performing visual inspection. In many instances, the operator's work load may be reduced by 
providing him with additional software tools such as automated scanning of the desired 
region, machine-vision inspection, and on-line flaw marking and annotation facilities. In the 
following, we will describe these inspection strategies in more detail. 
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Figure 5. Hardware Block Diagram of the Remote Surface Inspection System 


























4.1 Teleoperated Human Visual Inspection 

With human visual inspection, the operator inspects the surface through the camera 
monitors. If a flaw appears, he can stop the scanning and capture the close-up image of the 
object being viewed by the inspection camera and display it on the Close View window. The 
operator can then further examine the flaw, compare it with the ones marked in previous 
scans, and mark the flaw on-line, as will be discussed in more detail in the following 
subsection. 

In this mode of operation the operator interacts with the Graphical User Interface (GUI, shown 
in Figure 6) on the IRIS to set the appropriate mode of teleoperation and uses two joysticks to 
move the arm. The interface also allows the operator to move the arm by specifying the 
target position in the operator-commanded auto-move mode. One important feature of this 
system is that the operator can control the arm in shared control mode, which means that he 
can easily modify the preprogrammed or automated motions to avoid obstacles or to slow 
down the motion and prolong the inspection. Since the arm has seven degrees of freedom, 
a trigger on one of the joysticks can be used to control the elbow rotation to better position 
the arm and avoid obstacles. 

In auto-move mode the motion commands are generated by recording the arm positions in 
teleoperation and are stored in auto-sequencing scripts. These scripts can contain 
commands to perform other tasks such as image processing operations. The GUI also 
provides real-time graphical animation of the arm so that the operator can "view" the arm 
and its environment from directions not available using the actual camera. This capability is 
particularly useful when there is no direct viewing of the scene, as is the case for ground- 
based operations and for many on-board operations as well. Another window of the GUI 
displays digitized images that can be captured by any of the five cameras of the system. The 
operator can use either the camera monitors or the digitized images to inspect the object 
surface. 

4.2 Automated Scanning 

Object scanning can be made more efficient and reliable by an auto-scan planner. This 
allows the operator to simply designate an object or a region for which the system then 
automatically generates a scan path. Auto-scan planning includes the Far View, Close View 
and Object Definition windows shown in Figure 7. The Far View window displays the 
digitized image of the object being scanned. The image is captured from a camera at a great 
enough distance as to contain the entire object within the window. This window allows the 
operator to see the context of the area being inspected, while providing a means for mapping 
the object image to the actual object positions. 
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Figure 6. Graphics user interface to Remote Inspection System, 
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Figure 7. Automated Scanning Window. 



Object designation is achieved by moving the camera to each corner of the object with the 
desired inspection distance and registering each corner by reading the arm position and 
marking the corresponding corner on the image. If no image of the object is available, the 
Far View window shows a wire frame polygon with the corresponding corners. After 
mapping, a scan path can be generated which covers the whole area with overlap in each 
swipe. Two default scan paths can be chosen by the operator: the horizontal path which 
scans the object horizontally from top to bottom, and the vertical path which scans the object 
vertically from left to right. Additionally, via points where the arm will pass through, and vista 
points where inspection will be made, are generated along the path. The vista points are 
placed to ensure that the inspection will cover the whole object. More via points are put 
along the path to ensure smooth and accurate scanning. Moreover, the operator can use the 
menu to insert, delete, and relocate via points or vista points as needed for scanning objects 
with irregular shapes. Figure 7 shows a horizontal scan path with a point relocated. The 
object mapping and scan path generation are needed only when a new object is introduced 
to the system. The data are saved for subsequent inspection and can be loaded by selecting 
from a list of objects using the menu. The operator can then initiate scanning from the current 
arm position or any via points on the path, scanning either forward or backward along the 
path. He/she can also use the menu to request that the arm move to designated via points. 

4.3 Automated Scanning With Machine-Vision Inspection 

Before flaws can be detected automatically, a reference scan of the entire object is obtained. 
Flaw detection is achieved by comparing the images of the subsequent comparison scans 
against the ones from the reference scan. The operator selects the type of scan from the 
menu. The scan type and the list of vista points are sent to the Inspection Subsystem which 
performs automated inspection as described in the next section. The Inspection Subsystem 
signals the auto-scan planner when a flaw is detected. Normally, the arm is stopped and the 
operator marks the flaw just as he would with human visual inspection. The operator can 
also ignore the signal if he determines from the monitor that it is a false alarm. 

4.4 On-Line Flaw Manipulation 

In addition to automated scanning and inspection, the integrated environment provides on- 
line flaw manipulation for remote surface inspection. It allows the operator to input, save, 
retrieve, view, and compare the location, image, and annotation of flaws from previous and 
current scans. The operator marks a flaw by first placing the cursor on top of it in the Close 
View window and then using the menu to mark it. The program saves the flaw location, 
extracts its image to a file, and allows to operator to enter an annotation. Flaw marking and 
interpretation can be greatly enhanced by the ability to review the flaws from previous scans. 
This information is summarized in the scan history table. Each column, labeled with the date 
of the scan, contains the flaws marked in that scan. Each row, labeled by a flaw ID, contains 
the same flaws marked in different scans. Each entry on the table shows the location of the 
flaw. The operator can view a previous scan by double-clicking on the date of that scan. All 
the flaws marked in that scan are then shown on the Far View window in Figure 7. This 
allows him to determine if there are any flaws that are missing or newly added in the current 
scan. Using a cursor control device such a mouse or trackball, the operator can double-click 
on an entry to display the image of that flaw. He can also double-click on the flaw ID to 
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display all of the flaws with that ID. This allows him to see the changes of a particular flaw in 
each scan. The annotation of a flaw can be brought up by double-clicking its image. 


s. Automated Inspection 

Automated inspection is presently used in industry to inspect printed circuit boards, 
mechanical components, and other specific and well defined objects [10, 11, 12]. The 
objective of our research is to develop automated inspection techniques that can perform 
inspection for any general surface without adapting the algorithms for these objects. In the 
case of the SSF, we are interested in providing the operator with an automated inspection 
system which could be used for various ORUs, truss struts, pipes and extended surfaces 
such as radiators. Our initial approach is to survey the entire SSF by using a manipulator 
arm to collect images and other relevant data. Assuming a certain level of repeatability of the 
manipulator arm, it will be possible to re-survey the desired locations after a period of time 
and then compare the two sets of images. Whenever the system comes up with a large 
discrepancy between the ’before’ and ’after* images, it notifies the operator to confirm and 
log the flaw in a database. This simple image differencing technique provides a powerful 
inspection tool for the operator who only has to command the system and does not have to 
continuously visually inspect the images. This also provides an audible trace of previous 
inspection runs and findings. For this technique to work reliably, however, several technical 
issues must be resolved. 

First, any differences in the ambient light for the before and after images will introduce large 
discrepancies. Normally, these differences are large enough that a simple differencing 
technique always indicates possible flaws on the surface. The second problem is that of 
registration accuracy between before and after images. Even small errors in the position and 
orientation of cameras produces large discrepancies, particularly in high contrast areas of 
the images such as edges. The third problem is due to erosion of surfaces due to atomic 
Oxygen and ultraviolet exposure to the degree that changes in the overall reflectivity of the 
surface produce false triggers by the inspection system. In the following we will describe our 
approach to deal with the first two problems and discuss initial results. 

5.1 Ambient Light Removal 

Controlled lights mounted on the arm end-effector, base, and the fixed cameras can be used 
to cancel the ambient light effects. The concept is based on differencing two images of the 
same surface; one taken with the controlled lights off and the other with the lights on. The 
resulting image is an artificial image of the scene as if there were no ambient light. This 
image is stored as a “before image.” Similar operations are performed for the “after image 
and only then the before and after images are differenced. Figure 8 shows the result of an 
experiment for flaw detection under a variable ambient lighting condition. Note that in this 
scene, it is not easy to detect the missing screw even by human visual inspection. 
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Figure 8. Flaw Detection by Image Differencing 


5.2 Image Registration Accuracy 

Image differencing technique suffers from the inherent accuracy limitations of moving camera 
platforms. In the laboratory environment, i.e., fixed targets and industrial arms with good 
repeatability, the inaccuracy translates to no more that one to two pixels when images are 
taken from relatively short distances of less than 2 feet. In the space environment, we expect 
to have larger repeatability problems due to arm flexibility, large thermal variations, and 
object location changes due to thermal and structural flexibility of the structure. It is therefore 
necessary to develop techniques to account for the misregistration before comparing before 
and after images. 

We have devised several methods to estimate the registration error [13]. One approach is 
based on estimating the phase shift between the two images with spatial displacements in 
the frequency domain. It was found that this method can be used to reliably identify the 
lateral camera displacements (motion parallel to the image plane) in the presence of flaw 
and noise. This method is robust in the presence of noise but yields less accurate results 
when the image contains high frequency components. Figure 9 shows the before and after 
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images of the solar panel used to estimate the camera registration error. The white line in the 
second image represents a flaw. The dimensions are 200 X 300 pixels. The second image 
was displaced by -3 pixels in the horizontal and -2 in the vertical directions. The estimated 
shift was -2.25 and -1.9 in the horizontal and vertical directions, respectively. This results in 
sub-pixel registration correction which is the goal for the estimation process. 



Original image Displaced Image with Flaw 

Figure 9. Solar Panel Images Used for Camera Registration Estimation 

Two other approaches have been formulated and simulated for synthetic data. Application of 
these methods to real images are not complete. One approach is based on maximizing the 
correlation between the two images. The other is based on using the features on the images 
to estimate the camera position. 

5.3 Efficient Approaches to Automated Inspection 

Efficient automated inspection involves reliable flaw detection and robust flaw classification. 
The image differencing approach detailed above is clearly a useful first step in flaw 
detection. The method, however, does not address the important question of what scale the 
images should be processed at. The scale of an image is essentially the resolution 
associated with the image. It is well known [14] that relevant details of images exist only over 
a limited range of scale. Preliminary studies have been completed on estimating the image- 
scales relevant to flaw detection and classification. The essence of our approach is a 
multiresolution or "pyramidal" representation of the images. An initial high resolution image 
is successively convolved with a Gaussian kernel. The resulting images have decreasing 
spatial resolution and can be perceived as being stacked in a pyramid-like structure. The 
apex of this pyramid is a single pixel that represents the mean-pixel value of the original 
image which forms the base of the pyramid. The base level image, in addition to being noise 
prone, has a considerable amount of irrelevant information - that is the number of pixels 
associated with the image background is much greater that the number of pixels belonging 
to the image flaws. Efficient inspection, as we have learned from human inspection 
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strategies, involves discarding irrelevant information. Successive levels of the Gaussian 
pyramid from the base to the apex contain decreasing amounts of high frequency 
information. By taking the difference of two successive levels of a Gaussian pyramid, the 
"pass-band" content of the image can be estimated. This pass-band image is equivalent to 
performing an edge detection or a Laplacian convolution and reveals image structure 
information that is critical to flaw classification. A sequence of pass-band images, stacked to 
form a "Laplacian" pyramid reveals the range of scale that is most useful to flaw 
classification. 

Three classes of images are being used to experiment with the above technique. One class 
is made up of large (=2 m) field-of-view (FOV) pre-retrieval (in-orbit) LDEF surface images. 
These images were digitized from video sequences that were taped by the STS-32 shuttle 
crew during LDEF retrieval. The second set consists of medium (0.75 m) FOV images of 
laboratory ORU models. The third set consists of microscopic FOV (1 mm) images which 
were obtained from the LDEF post-retrieval image database at Johnson Space Center. 


6. Experimental Results on Human Visual Inspection 

An experiment was conducted to evaluate the usefulness of the system for the detection of 
damage caused by micrometeoroids [15]. A typical tank ORU was used for this task. The 
specific objective was to determine the time-to-completion and accuracy of inspection for 
micrometeoroid impact causing damage ranging from 1 to 10 pixels on the surface of the 
ORU. As part of the evaluation, we also compared the telerobotics-based inspection against 
direct human inspection. Since direct human visual inspection is unencumbered by the 
helmet that obscures EVA inspection, this provides a worst case test for telerobotics 
inspection. Table 3 shows the results of these experiments for the case of Teleoperated 
Human Visual Inspection discussed in Section 4.1. These preliminary tests show that remote 
inspection is approximately three times slower and 3 to 4% less accurate than direct 
inspection These results indicate that remote inspection can provide a safe and effective 
alternative to EVA inspection for a class of tasks. 


Table 3. Experimental Results for Remote and Direct Mlcrometeorold Inspection 


Flaw Size 

Remote Surface 
Inspection 

Direct Inspection 

Large Marks, 10 Pixel 
(2.7 Mm) 

Time-To-Completion: 
178 Sec 
Accuracy: 

93% 

Time-To-Completion: 
57 Sec 
Accuracy: 

97% 

Small Marks, 1 Pixel 
(0.27 Mm) 

Time-To-Completion: 
308 Sec 
Accuracy: 

91% 

Time-To-Completion: 
1 18 Sec 
Accuracy: 

94% 
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7. Conclu sions and Future Work 

This Daper has described the research and development effort for remote surface inspection 
at the Jet Propulsion Laboratoiy which started in 1991. The paper outlines the problem, the 
general approach and present initial results. 

Future work will involve adding other manipulation and inspection sensors to the existing 
system for collision avoidance and for the detection of flaws which cannot be inspected by 
CCD cameras, such as gas leak, fine cracks, temperature variations, and so on. 


The differencing approach will be further developed to use a scanning technique that will 
perform flaw detection continuously in real-time without stopping the arm to take images. 
Technique to Accommodate large misregistrations and to categorize flaw type are also being 
implemented. 
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